A Formal Scheme for Multimodal Grammars
نویسندگان
چکیده
We present in this paper a formal approach for the representation of multimodal information. This approach, thanks to the to use of typed feature structures and hypergraphs, generalizes existing ones (typically annotation graphs) in several ways. It first proposes an homogenous representation of different types of information (nodes and relations) coming from different domains (speech, gestures). Second, it makes it possible to specify constraints representing the interaction between the different modalities, in the perspective of developing multimodal grammars.
منابع مشابه
Pattern Grammars in Formal Representations of Musical Structures
This paper introduces several formal models of pattern representation in music. Polyvalent multimodal grammars describe partially overlapping sound events as found in polyphonic structures. Bol Processor grammars are characterizations of sequential events in terms of substring repetitions, homomorphisms, etc. Parsing techniques, stochastic production and recent developments of BP grammars are b...
متن کاملMultimodal Dialogue System Grammars∗
We describe how multimodal grammars for dialogue systems can be written using the Grammatical Framework (GF) formalism. A proof-of-concept dialogue system constructed using these techniques is also presented. The software engineering problem of keeping grammars for different languages, modalities and systems (such as speech recognizers and parsers) in sync is reduced by the formal relationship ...
متن کاملThe Turing Completeness of Multimodal Categorial Grammars
In this paper, we demonstrate that the multimodal categorial grammars are in fact Turing-complete in their weak generative capacity. The result follows from a straightforward reduction of generalized rewriting systems to a mixed associative and modal categorial calculus. We conclude with a discussion of a restriction to the so-caled weak Sahlqvist lexical rules, for which we can ensure decidabi...
متن کاملComputationally secure multiple secret sharing: models, schemes, and formal security analysis
A multi-secret sharing scheme (MSS) allows a dealer to share multiple secrets among a set of participants. in such a way a multi-secret sharing scheme (MSS) allows a dealer to share multiple secrets among a set of participants, such that any authorized subset of participants can reconstruct the secrets. Up to now, existing MSSs either require too long shares for participants to be perfect secur...
متن کاملQuantitative Constraint Logic Programming for Weighted Grammar Applications
Constraint logic grammars provide a powerful formalism for expressing complex logical descriptions of natural language phenomena in exact terms. Describing some of these phenomena may, however, require some form of graded distinctions which are not provided by such grammars. Recent approaches to weighted constraint logic grammars attempt to address this issue by adding numerical calculation sch...
متن کامل